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ABSTRACT 

A  master's  thesis  which  discusses  the  solution  of  con- 
cave-convex games.   An  algorithm  is  developed,  a  computer 
program  written  and  applied  to  an  anti-submarine  warfare 
force  allocation  problem  as  an  illustration.   Techniques 
for  handling  concave -convex  problems  in  high  dimensions 
are  included. 
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I.   INTRODUCTION 

A.    A  GAME  THEORY  APPROACH  TO  RESOURCE  ALLOCATIONS 

Problems  in  the  allocation  of  resources  can  be  divided 
into  two  descriptive  categories:   Single-agent  problems  and 
adversary  problems.   Single-agent  problems  have  only  one 
participant  optimizing  without  intelligent  opposition.   In 
the  solution  of  adversary  problems,  opponents  work  at  cross 
purposes.   Each  choice  of  an  allocation  of  resources  by  one 
participant  must  be  made  in  light  of  those  of  his  opponent  (s). 
Of  course,  games  are  adversary  problems. 

The  great  interest  in  game  theory  as  a  technique  of 
modelling  which  followed  von  Neumann's  statement  of  the 
fundamental  concepts  in  19Z7  [ISj  ^oiilijiues  today.   The 
fascination  of  the  game  as  a  model  for  conflicts  of  almost 
any  sort  is  enhanced  by  the  fact  that  the  solution  to  a 
game  is  entirely  independent  of  assumptions  regarding  the 
actual   behavior  of  the  antagonist (s) . 

Matrix  games  and  differential  games  are  extensively 
treated  in  the  literature;  some  references  are  noted  here. 
Matrix  games  or  games  over  the  square  are  discussed  in 
basic  form  by  Williams  [21]  and  a  thorough  study  is  done  by 
Karlin  [8].   For  a  first  essay  in  the  subject  of  differen- 
tial games,  see  Isaacs  [6].   Taylor  [16,17]  gives  additional 
examples  of  the  modelling  of  combat  operations  including 
search  using  differential  games. 

The  development  of  the  derivative  game  is  due  to 
Danskin  [4].   A  very  large  class  of  concave-convex  problems 


yield  to  solution  when  the  algorithm  based  on  this  game  is 
applied.  It  is  with  the  derivative  game  and  the  algorithm 
evolved  from  it  that  this  paper  is  primarily  concerned. 

B.    THE  CONTENT  OF  THE  PAPER,  BY  CHAPTER 

Chapter  II  surveys  some  of  the  more  important  mathe- 
matical ideas  necessary  to  the  development  of  the  algo- 
rithm for  the  solution  of  concave-convex  games. 

Chapter  III  is  concerned  with  the  programming  of  an 
anti-submarine  warfare  force  allocation  example.   The  ex- 
ample serves  to  illustrate  both  the  use  of  the  algorithm 
and  the  theory  on  which  it  is  founded.   The  problem  is 
formulated  and  the  basic  computational  steps  toward  the 
solution  are  enumerated.   If  any  fears  stem  from  the 
formidable  appearance  of  the  matrices  used  in  the  example, 
they  hopefully  will  be  dispelled  by  the  description  of  the 
form  for  input  in  the  programming  notes.   Technical  matters 
regarding  programming  are  considered  in  some  detail. 

Chapter  IV  draws  together  this  presentation  with  some 
concluding  remarks . 

Chapter  V  contains  suggestions  for  further  study. 

The  Appendix  consists  of  the  programming  flowchart. 
The  Computer  Program  Listing  is  included  as  well. 


II.   MATHEMATICAL  CONCEPTS 

A.    GENERAL 

The  fundamental  theorem  of  the  theory  of  games  in  the 
form  appropriate  here  is  the  following: 

Suppose  F(x,y)  is  a  continuous  function  on  X  x   J, 
where  X   and  Y   are  compact  and  convex.   Suppose  that  the  set 
of  points  X(y)  yielding  the  maximum  to  F  for  fixed  y  is 
convex  for  each  such  y,  and  that  the  set  of  points  Y(x) 
yielding  the  minimum  to  F  for  fixed  x  is  convex  for  each 
such  x. 

Then  there  exist  pure  strategy  solutions  x°  and  y° 
satisfying 

A   1  .'V    -  V  I     '  A  I  A       *r  !      T7     v    <,  J. 

(1) 

F(x,y°)  <  F(x°,y°),  V  xeZ. 

A  complete  proof  of  the  theorem  in  this  form  can  be  found 

in  [7].   Assuming  that  the  conditions  for  the  existence  of 

a  solution  satisfying  (1)  are  met,  the  problem  can  be  stated 

in  the  form 

Max  Min  F(x,y) , 
x   y 

which  is  equivalent  to 

Max  cj)(x) 
x 

where  <KX)  -  Mi-n  F(x>y)- 

y 

The  theorem  applies  to  two -person  zero-sum  games.   Thus, 

Max  Min  F(x,y)  =  Min  Max  F(x,y). 
x   y  y   x 


One  important  difficulty  arises  from  the  fact  that, 
although  F  may  be  smooth,  cf> (x)  is  not  in  general  differ- 
entiate in  the  ordinary  sense.  Danskin,  in  [3],  has  shown 
that  under  general  conditions  on  X  and  Y,  there  exists  a 
directional  derivative  in  every  direction.  It  is  this  fact 
that  has  provided  the  key  to  the  solution  of  problems  of 
the  type  described. 

B.    THE  ALGORITHM  FOR  THE  SOLUTION  OF  CONCAVE -CONVEX  GAMES 
The  algorithm  to  solve  games  concave  in  the  maximizing 
player  and  convex  in  the  minimizing  player  is  developed  and 
presented  in  great  detail  in  [4].   The  following  gives  a 
brief  survey  of  those  results  which  are  most  important  for 
the  design  of  the  algorithm;  to  fill  the  apparent  gaps,  a 
thorough  reading  of  [4]  remains  necessary. 
1 .   The  Derivative  Game 

Suppose  x° cX .      Associate  with  x°  a  non-empty  set 
of  admissible    directions    y>  r(x°) .   W  is  defined  as  the 
convex  hull  (e.g.,  see  [19])  of  the  set  of  points 

W(y)  =  {F    (x°,y),  -••  ,  F    (x°,y)}, 
xl  yk 

where   yel(x°)  . 

The  derivative  game  then  is 

H  (  y  ,  w  )  =  y  *  w 
defined  over  r(x°)  x   W.   The  maximizing  player  maximizes  H 
by  choice  of  yeT(x°),    the  minimizing  player  minimizes  H  by 
choice  of  weW. 


THEOREM  I 

A  necessary  and  sufficient  condition  for  the  exis- 
tence of  a  direction  of  increase  for  <J>(x)  is  that 
the  value  of  the  derivative  game  defined  by  II  be 
positive  at  x° .   The  y°  which  yields  the  value  of 
the  derivative  game  is  a  pure  strategy. 
If  the  value  is  positive  one  can  find  and  use  this  direction. 
If  the  value  is  non-positive,  a  direction  of  increase  does 
not  exist,  i.e.,  the  solution  has  been  reached. 
The  application  of  the  derivative  game  in  practice  is  greatly 
complicated  by  the  necessity  for  approximations. 
2 .   The  Lemma  of  the  Alternative 

The  Lemma  takes  into  exact  account  the  approxima- 
te ons  involved  in  the  application  of  the  derivative  game. 
It  states  that  a  certain  process  (to  be  explained  below) 
must  either  yield  a  sufficient  increase  to  cj)(x)  at  a  point 
x°  or  determine  that  the  point  x°  is  nearly  optimal.   Be- 
fore the  lemma  can  be  formulated  in  mathematical  terms, 
some  further  difficulties  and  the  tools  with  which  to  over- 
come them  have  to  be  outlined. 

a.   The  Brown-Robinson  Iterative  Process  in  the 
"Auxiliary  Game" 

The  Brown- Robinson  (B-R)  process  employs  the 

following  idea:   Let  G  be  the  pay-off  function.   At  stage 

N=0  both  players  choose  arbitrary  strategies  x°  and  y° .   At 

stage  N=l  the  maximizer  chooses  x1  such  that  G  is  maximized 

against  y° ;  then  the  minimizer  chooses  y1  to  minimize  G 


against  x1 ,  and  so  forth.   At  stage  N  the  maximizer  chooses 
x   as  if  the  minimizer ' s  strategy  were  an  evenly  weighted 
mixture  of  strategies  y°  ,  •••  •  ,y]   ;  the  minimizer  chooses 
y   as  if  the  maximizer' s  strategy  were  an  evenly  weighted 
mixture  of  strategies  x° ,  •••  ,x  . 

I  For  matrix  games,  Julia  Robinson  [12]  proved 

that 

IN  "  1  JN 


lim  sup 
N-*-°° 


r   N-l  N 

m      n=0 


n=0 


Danskin  [1]  has  generalized  the  proof  to  hold 
for  two -person  zero -sum  games  with  continuous  pay-off  de- 
fined over  X  x   y,  x   and  Y   arbitrary  compact  spaces.   It 
should  be  noted  here  that  the  B-R  process  is  very  slow  in 
convergence  when  applied  directly  ^o  finding  an  approxims 
tion  to  the  value  of  the  game  defined  by  (1).   However,  it 
is  not  applied  to  the  basic  game  in  this  algorithm  but  rather 
to  an  "auxiliary  game"  for  which  an  accurate  solution  is  not 
required. 

The  derivative  game  mentioned  above  cannot  be 
solved  directly  because  the  set  Y(x°)  is  not  known.   All  one 
has  is  a  single  element  ycY   which  approximately  minimizes 
F(x°>y)-   The  place  of  the  derivative  game,  therefore,  is 
taken  by  the  "auxiliary  game"  employing  a  modified  version 
of  the  B-R  process  described  below.   This  process  makes  it 
possible  to  keep  track  of  the  approximations  involved  and 
their  consequences.   The  "auxiliary  game"  is  defined  as 
follows : 
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For  any  £>0,  denote  by  Y  (x)  the  set  of  ycY 
such  that  F(x,y)  =  <f>(x)  +  e-   Let  Y  (5)  =  U  Y  (x)  ,  yeT^x0), 

yeY(E). 

E(x°+d  Y,y)  -  <Kx°) 

Then  H(Y,y)  =   ^ ,  for  dQ  >  0, 

o 
a  minimum  step  size,  is  a  game  over  r(x°)  x   Y  (5)  with  y 

the  maximizing  and  y  the  minimizing  player.   This  game  has 

optimal  mixed  strategies  for  both  players.   Applying  the 

idea  of  approximate  optimization  to  the  convergence  proof  in 

[1]  leads  to 


THEOREM  II 

N 


N-l 


n 


Let  y  be  chosen  such  that  ^     En  H(y,y  )  is 
maximized  to  accuracy  c,  and  y  be  chosen  such 
that  rr  Z^  H(y  ,y)  is  minimized  to  accuracy  n . 


i  lien 


lim  sup 


\\   H(Y)yn)  -  I     En  H(Yn,y) 
n=0  n=0 


<  2(c+n) 


This  holds  for  continuous  H 


Let  6  be  the  maximum  oscillation  of  VF(x,y)  over  a  distance 


d  .   Then 
o 


F(x°+do  YN,yn)  -  F(x°,yn)  >  yN  F(x°,yn)  -  e 
_ 

o 
The  lemma  of  the  alternative  now  can  be  formulated, 
b.   Statement  of  the  Lemma 

Suppose  0<a<3,  y°eY  (x°). 
Then  the  generalized  B-R  process  will,  at  some  stage  N,  de 
termine  that  one  of  the  two  following  statements  is  true: 

1.   The  maximum  over  F(x°)  of  the  directional 
derivative  does  not  exceed  3. 
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N  -N 


2.   The  point  x   =  x°  +  d  y    »    where 

N 


-N_l   £   n 


and  the  point  y  eY  (x  ) ,  where  y  minimizes 
F(x  ,y)  to  accuracy  £,  satisfy 

F(xN,yN)  -  F(x°,y°)         3 

3  >  a  bo  ^-  . 

o  o 

3.   The  Corollary  of  the  Alternative 

Suppose  that  F(x,y)  is  concave  in  x  and  convex  in 
y.   Then  the  modified  B-R  process  applied  at  x°  will,  at 
some  stage  N,  determine  that  one  of  the  two  following  state- 
ments is  true : 

1.  The  pair  x  =  x° ,  y  =  y  ,  where 

-N   _i_  <£   n 

y    N+l  L    y  ' 
n=0 

are  approximate  optimal  strategies  for  the  game 

defined  by  F. 

2.  The  point  x  zX   yields  an  increase  to  <J>(x)  by  at 
least  a  specified  amount. 

For  details  and  proof  see  [4],  pp.  36  ff. 

Reference  [4]  continues  with  a  detailed  discussion 
of  delicate  problems  which  can  only  be  listed  here:   The 
choice  of  the  minimal  step  size  d  ;  the  problem  of  acces- 
sibility of  a  point  x  from  a  point  x°  ;  the  problem  of  ob- 
struction; the  choice  of  a,  3,  c,  their  interaction  with  each 
other,  and  the  choice  of  p  where  p  is  the  accuracy  to  which 
Max  <t>(x)  is  to  approximate  the  value  of  the  game  defined  by 
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(1) .   It  must  be  noted  that  the  conditions  derived  for  the 
selection  of  these  parameters  are  sufficient. 
4 .   The  Algorithm 

The  algorithm  as  a  consequence  of  the  foregoing 
mathematical  considerations  is  presented  in  section  10  and 
11  of  [4]  and  will  not  be  reproduced  here  in  detail.   A  ver- 
bal description  of  its  basic  structure  -  depicted  in  Figure 
1  -,  however,  may  be  useful: 

The  maximizing  player,  called  Max,  having  arrived 
at  a  point  xx ,  has  a  direction  of  maximal  increase  y,  ob- 
tained either  from  the  derivative  game  D  cf>(xx)   or  from 
the  B-R  process  in  the  auxiliary  game,  and  a  distance  d  >  d  . 
The  minimizing  player,  called  Min ,  is  at  a  point  yy.   F(xx,yy) 
is  known.   Max  makes  a  proposal  uo  move  to  a  point  x  =  xx  +  dy 
Of  course,  F(x,yy)  >  F(xx,yy).   Min  accepts  Max's  proposal 
and  starts  minimizing  against  x,  looking  for  a  direction 
of  maximal  decrease  g.   If  there  is  none,  yy  is  a  minimum 
against  x  as  well  as  against  xx  in  which  case  Max  will  move 
to  the  point  x.   If  there  is  a  direction  of  decrease  Min 
forms  a  point  y  =  yy  +  Dg  such  that  F(x,y)  <  F(x,yy).   A  test 
is  performed  to  determine  whether  Min  has  already  "beaten" 
Max:   if  F(xx,yy)  >  F(x,y)  Min  stops  the  minimization  pro- 
cess, and  Max  discards  his  proposal  x  because  moving  to  x 
will  not  increase  <KX)  •   Max  halves  the  distance  d  and,  with 
the  same  y,  forms  a  new  trial  point  x.   If  F(xx,yy)  <  F(x,y) 
Min  continues  to  minimize  until  either  F(xx,yy)  >  F(x,y)  or 
Min  can  no  longer  find  a  direction  of  decrease.   If  now 
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Figure  1.   Flowchart  of  Algorithm. 
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F(xx,yy)  <  F(x,y) ,  indicating  that  Min,  even  after  a  complete 
minimization,  was  not  able  to  "beat"  Max,  Max  moves  to  the 
proposed  point  x  realizing  a  gain  for  <KX)  •   Max  then  looks 
for  a  new  direction  of  increase.   The  process  terminates  when 
such  a  direction  does  not  exist. 

The  situation  that  leads  into  the  Brown-Robinson 
process  is  the  following  one:   The  proposed  points  x  =  xx+dy 
have  been  "beaten"  by  Min  until  d  gets  cut  down  to  d  ,  in 
spite  of  the  fact  that  y,  obtained  from  the  derivative  game, 
is  an  apparently  good  direction.   If  the  trial  point  x  =  xx 
+  d  y  does  not  result  in  a  move  for  Max,  the  B-R  process, 
Figure  2,  is  used. 

Denote  the  present  y  -  the  one  that  so  far  has 

lead  to  a  failure  rc-r   r.:nx    by  y " ,  s^d  liit;  associated  VF 

'  '  '  x 

by  to°  .   Minimize  F  completely  against  x  =  xx  +  d  y .   The 
resulting  y  then  leads  to  a  new  VF  which  is  averaged  with 
the  previous  to0,  giving  to1.   Suppose  that  to 1    as  input  to 
the  derivative  game  D  4>(x)  produces  a  new  y°  such  that  the 
value  of  the  derivative  game  is  positive  as  required.   (If 
such  a  y°  does  not  exist  the  problem  is  solved).   This  y, 
averaged  with  y1,  gives  y2  which  in  turn  creates  a  new 
trial  point  x2  =  xx  +  d   y2.   x2  ,  or  x1  in  general,  then  is 
exposed  to  Min ' s  reaction  as  described  previously.   Once  Max 
finds  a  direction  and  an  associated  trial  point  that  cannot 
be  "beaten"  by  Min,  Max  moves  and  leaves  the  B-R  routine. 
The  y-strategies  are  also  averaged  and  saved  although  their 
average  is  never  used  during  the  computation.   In  case  the 
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Figure    2.      The    Brown-Robinson   Process. 
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game  defined  by  F(x,y)  is  terminated  while  in  the  B-R  pro 
cess,  this  average  of  the  y-strategies  represents  the  op- 
timal solution  for  the  minimizing  player. 
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Ill .   PROGRAMMING  ASPECTS  OF  AN  ILLUSTRATIVE  EXAMPLE 

A.    FORMULATION  OF  THE  PROBLEM 

An  algorithm  for  the  solution  of  a  wide  class  of  con- 
cave-convex games  over  polyhedra  has  been  presented  in 
abbreviated  form  above.   In  this  chapter  that  algorithm  is 
applied  to  a  particular  game,  an  anti-submarine  warfare 
force  allocation  problem. 

For  this  problem,  five  indices  are  employed:   h,  i,  j, 
k,  and  m.   In  a  meaningful  example,  the  maximum  numbers 
corresponding  to  these  indices  might  be,  respectively: 
5,  25,  10,  500,  and  5.   The  indices  have  the  following 
meanings : 

):■  ;   Submarine  type 


.1 

k 


Submarine  mission 

Type  of  antisubmarine  weapon  (or  vehicle) 

Place  in  which  submarine  and  weapon  encounter  one 

another 


m:   Stage  of  the  submarine  mission. 
An  additional  index  is  used.   &(k)  is  the  "kind  of  place." 
A  "kind  of  place"  might  be  defined  by  a  particular  set  of 
weapon  employment  parameters.   These  include  the  tactical 
and  the  natural  environment.   The  natural  environment  con- 
sists of  oceanographic  and  meteorological  conditions.   Ex- 
amples of  the  tactical  environment  are  destroyers  in  an  ASW 
screen  and  patrol  aircraft  in  barrier  patrol.   The  "kind  of 
place"  in  which  an  encounter  occurs  impacts  on  the  outcome 


18 


of  an  encounter  between  submarine  and  weapon.   A  reasonable 

number  of  "kinds  of  places"  in  the  present  context  might  be 

25. 

A  submarine  mission  is  described  by  a  matrix   |EU . .,   I . 

'  ' '  hijkm ' ' 

This  matrix  has  as  its  elements  real  numbers  denoting  the 
extent  to  which  a  submarine  of  type  h  at  stage  m  of  mission 
i  is  exposed  to  a  weapon  of  type  j  at  the  kth  place.   The 
effects  of  these  weapons  and  thus  of  the  encounters  are 
characterized  by  a  "technical"  matrix   l^hi£fk")l   ^n  t^ie  ^°^" 
lowing  meaning:   exp[-C,  .{  ,,  >.  ^'V^  ^-s  tne  probability  that 
a  submarine  of  type  h  survives  one  exposure  to  y  units  of 
weapons  of  type  j  at  a  "kind  of  place"  £(k).   These  en- 
counters are  assumed  to  be  mutually  independent.   Note  that 
the  probability  of  survival  to  the  m^n  mission  stage  is 
conditioned  on  the  completion  of  the  previous  stages.   Sup- 
pose that  there  are  y.,  units  of  force  of  type  j  at  the  ktn 

3  K 

place.      Then   the   probability   of   a   submarine's    completing 


stage   m   of   mission    i    is 


7|     exp[-Ehijkm,    ChjA(k)    yjk], 


m'  <m 

which,   due  to  independence  of  the  events,  equals  exp  [-0,  .  ] 
where 

6him   V  *g   Ehijkm'  Chj£(k)  yjk* 

m '  <m 

Now,  by  carrying  out  a  premultiplication , 


A      =  E      C 
hijkm    "hijkm  hj£(k)' 
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the  exponent  becomes 

6him  =  Z    £  Ahijkm'  yjk' 
in'  <m 

In  this  example,  the  vast  majority  of  the  E,  .  . ,  ,  and  there 

fore    of   the   A,  ..-,     ,    are    zero.       If    3000   non-zero   A,..,       are 

hi j km  hi j km 

allowed,  each  type  of  submarine  can  be  employed  on  ten  dif- 
ferent missions  and  undergo  up  to  60  encounters  with  anti- 
submarine weapon  systems. 
1 .   The  Space  X 

Let  x,  .  be  the  proportion  of  submarines  of  type  h 
assigned  to  the  i"th  mission.  Make  X  =  |  |  x,  .  |  satisfy  the 
conditions 


;OT  pvprv  h ,  x,  . 


.hi      '  '   hi  - 

1 


and 


ahi  -  xhi  -  3hi'  £or  ever>'  Pair  h'i' 
where  the  sets  {a,  •},  (Bv-1   are  supposed  to  satisfy 


L,   a,.  <  1  <  L,   3,  •  for  every  h 


and 


0    <   a,  -    <    3^  •    for   every   pair   h,i. 

2 .       The    Space    Y 

Let  y . ,  be  the  proportion  of  antisubmarine  forces 
of  type  j  sent  to  the  k™  place.  Make  Y  =  l>'-vl  satisfy 
the  conditions 
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and 


L     y-k   =    1,    for   every   j,    y^k    >    0 

JK 


ajk    -  yjk    -   bjk   £or   ever>r  Pair   J>k 


where  the  sets  {a.-,  },  {b., }  are  supposed  to  satisfy 

L     a.,  <  1 ,  <  L  b  ., 
k   Jk        k    Jk 


and 


0    <    a.-,     <   b . ,     for   every   pair   j  ,k 


3 .   The  Function  F(x,y) 

V,  •  is  the  value  of  accomplishing  stage  m  of 
mission  i  for  a  submarine  of  type  h.  The  character  of 
F(x,y)  can  be  examined. 


06  u 


w 


,  -  =  v,  .  exp  r-e,  .  ] 

him    him   *    L   himJ 


where  8,  .   is  as  before.   Put 
him 


hi     *-*       him 


m 

Then 

m 

F(x,y)  =  7  xhi  Thi. 

h.,i 

This  function  is  linear  in  x  and  exponential  in  y  and  is 

therefore  a  concave -convex  game  of  the  type  treated  in  [4], 

defined  over  X  x   Y.   The  quantity  F(x,y)  represents  the 

total  expected  payoff  to  the  submarine  player.   Re-expressing 

F(x,y)  in  its  explicit  form  gives: 
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F(x,y)  =  L,     l,     x,  .  A,  V,    exp  [  -  /.  ),  E,  .  .-,   C,  ..  n  sV.,  1, 
v  ,7^     i   -    hi  ^   him   v  l     v  T   hiikm  h;i£(kyikJ 
111       m  J   k      J      j  *,  ;  j 

The  remainder  of  this  chapter  gives  details  of  the  application 
of  the  algorithm  to  the  game  defined  above. 

The  principal  result  is  the  flow-chart  (Appendix). 
Since  this  flow  chart  is  constructed  around  the  algorithm 
from  [4],  it  is  helpful,  though  not  essential,  to  have  [4] 
available.   The  complete  program  listing  is  included  following 
the  Appendix.   The  program  is  written  in  FORTRAN  IV  and  was 
run  on  the  IBM  360/67  computer  at  the  Naval  Postgraduate 
School,  Monterey,  California. 

B.    BASIC  COMPUTATIONS 
1 .   C \_>  nip  u  L  a 1 1  o ii  v  f 


mm 


For  fixed  (h,i),  9,  .   is  non-decreasing  in  the  mis- 

v  '  J  '   him  fa 

sion  stage  m.   This  reflects  the  trivial  fact  that 

P [submarine  survives  stage  m] 
<  P [submarine  survives  stage  m-1]. 

Equality  holds  when  the  "threat"  due  to  the  encounters  at 
stage  m  is  non-existent,  i.e.,  when  either  no  ASW-forces 
y.,  are  present  or  their  effectiveness  against  the  submarine, 
^h'&fkV  ^"s  zero-   Hence  0,  .  ,  for  each  pair  (h,i),  is  ac- 
cumulated over  the  mission  stages  as  follows: 

6,  .   =  G,  .r   ,.  +  V    V     A,  .  .,   y .,  ,  9,  .   =  0 . 
him    hi  (m-1)    h*    ^       hijkm  7jk'   hio 
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2.  Partial  Derivatives  with  Respect  to  x,  . 
hi 

Because  of  the  linearity  of  F  in  x  the  partials  with 

respect  to  x,  . ,  F    ,  are  the  coefficients  of  x,  .  and  do  not 
ilJ-   Xt  z  Jii 

explicitly  contain  x.   V}iim  exP[-6}lim]  can  be  represented  as 
a  matrix  of  dimension  (n  x   m) ,  where  n  is  the  number  of  pairs 
(h,i).   Then 

Fv    =  V   V,  .   exp[-0,  .  ] 
x,  .     *->     him    ^L   himJ 

is  the  sum  of  the  elements  in  the  (h,i)  row  of  that  matrix. 

3.  The  Value  of  F(x,y) 

F(x,y)  is  obtained  by  pre-multiplying  F     by  x,  .  and 

Xhi      ni 
summing  the  products 

F(x,y)  -   £  x,.  Fv_  . 


n  i 


4.   Partial  Derivatives  with  Respect  to  v., 
*        7  -}  K 

Because  of  the  cumulative  property  of  6,  .  ,  a  change 

in  y-k  during  some  mission  stage  m '  will  affect  the  following 

stages  as  well.   For  each  pair  (h,i),  premultiply     x,  .  by 

the  corresponding  Ahi-jkm,>  where  m'  is  the  mission  stage  in 

which  the  y.,  of  interest  occurs.   Sum  V,  .   expf-G,  .  1  over 
1 K  him    '  L   himJ 

the  mission  stages  for  which  m ' <m ,  multiply  the  result  with 
Xhi  Ahiikm'  and  sum  tne  Products  over  h  and  i: 

'\k~~     I     ?  Xhi  AhiJkm.  ra^  Vhim.  exp[-9him,]. 

5 •   Finding  Directions  of  Increase  (Decrease) 

F    and  F    are  inputs  to  the  derivative  games  for 
hi      }ik 
x  and  y.   For  x,  the  direction  y°  is  sought  such  that  D  <j>(x)  = 

VFx«y  is  maximized;  for  y,  g°  is  sought  such  that  D  Max  F(x,y) 

g   x 
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VF  »g  is  minimized  (equivalently ,  -VF  «g  is  maximized).   The 
side  condition  is  that  y°  and  g°  be  unit  vectors: 

i       k   J  1        k   J 

A  method  of  finding  such  directions  -  called  THE  DIRECTION 
FINDING  ALGORITHM  -  is  derived  from  the  Kuhn-Tucker  conditions 
and  the  Schwartz  Inequality.   It  is  contained  in  [4]  and  has 
been  applied  to  a  vector  -  valued  function  by  Zmuida  in  [22]. 

C.    PROGRAMMING  NOTES 

The  flow  chart  (Appendix)  and  the  associated  program  may 
not  be  optimal  with  respect  to  machine  time  and  memory  space 
required,  and  may  provide  opportunities  for  improvement.   For 
computer  routines  like  this,  intended  to  solve  high-dimensional 
problems,  a  trade-off  between  time  and  space  is  generally  ap- 
parent.  The  user,  considering  the  particular  facilities 
available  to  him,  must  decide  on  his  optimal  trade-off,  and 
modify  the  program  accordingly.  The  following  discusses  some 
of  the  techniques  that  have  been  implemented;  it  points  out 
the  major  difficulties  that  have  been  encountered  and  the  ways 
presently  used  to  deal  with  these,  and  offers  some  remarks 
about  the  impact  of  the  underlying  mathematics  on  the  use  of 
the  algorithm  for  realistic  problems.   The  numbers  referenced 
are  statement  numbers. 

1 .   Presentation  of  Input  -  The  Mission  Description 

In  a  realistic  application,  most  of  the  A,  •  •  •,,   will 

be  zero  because  the  effectiveness  C,  . „ r, ^  contained  in  A  will 

hj  £(k) 

be  zero.   Example:   suppose  k  denotes  a  place  in  the  western 
Baltic  and  &(k)  classifies  this  place  as  shallow  with  extreme- 
ly poor  sonar  conditions;  j  denotes  a  nuclear  killer-submarine, 

1  A 


h  represents  a  conventional  attack  submarine.   Then  C,  .  .  ,,. 
will  be  zero  for  all  practical  purposes. 

The  method  of  presenting  the  matrix  E  which  avoids 
computing  of  and  with  A  when  C  is  zero,  is  one  which  is  ex- 
tremely easy  for  the  user  to  understand  and  employ.   He  gives 
his  description  of  a  submarine  mission  by  plotting  it  on  a 
chart,  marks  off  in  order  the  places  the  submarine  must  go, 
and  notes  the  forces  it  might  meet  at  those  places.   He 
will  give  the  exposure  E  required  by  the  particular  mission 
in  terms  of  a  standard  which  he  will  have  set.   He  will  mark 
off  the  points  in  the  mission  where  the  various  stages  of 
the  mission  will  have  been  accomplished.   Such  a  description 
might  run  as  follows: 

m=0 


1%  —  1     A   —  1     1/-—  "1     A  —  0  "B  —  '   M 

a  -  _L     1-1     i\  ~  X     J  ~  ^     J-.  —  X  .  U 

j=5   E=1.5 

k=7   j  =  l   E=2.0 

j  =  3   E=3.0 

m=l  (the  first  stage  of  the  first  mission  is  completed) 

k=8  j=4  E=1.2 
j=7   E=1.7 

m=2 

k=15  j  =  3  E=0.3 
j=2  E=2.0 
j=4   E=0.5 

m=3 

The  above  mission  (mission  1  for  submarines  of  type 
1)  has  three  stages  and  nine  encounters,  in  four  different 
places.   Then  the  listing  starts  for  the  next  mission  with 
h=l,  i=2,  and  continues  until  all  missions  for  all  types  of 
submarines  have  been  described  in  this  manner.   Note  that 
this  listing  has  already  taken  into  account  the  classification 
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of  the  places  k  by  £(k).   This  is  done  in  such  a  way  that 
for  each  encounter  on  this  mission  listing  the  associated 
C,  •  p  s,  s  is  positive.   In  other  words,  a  place  k,  and  hence 
an  encounter,  will  appear  on  the  mission  listing  only  if  it 
seems  possible  that  the  opponent  will  allocate  forces  to  that 
place  and   the  corresponding  effectiveness  against  the  maxi- 
mized s  submarine  operating  in  that  place  is  positive. 

There  are  various  possible  ways  of  storing  the 
information  contained  in  the  mission  description  in  the 
machine.   One   way  which  is  very  economical  in  terms  of 
storage  space  requirements  is  to  read  the  list  encounter  by 
encounter,  i.e.,  card  by  card,  starting  with  an  N=l.   Then 

A(N)    =  E'C,  .„,n  and  an  indicator  array 
h3&(k)  ; 

P(N)  =  I-J-K-M- (h-1)  +  J-K-M-(i-l)  +  K-M-(j-l) 
+  M« (k-1)  +  m  +  1 

contain  the  complete  information.   I,  J,  K,  M  denote  the 
maximum  values  of  the  indices  i,  j,  k,  m.   The  reason  for 
using  (m+1)  is  that  the  correct  m  appears  after   the  mth 
stage  is  completed.   The  disadvantage  of  this  method  is 
that  during  computations  the  individual  indices  must  be  re- 
computed from  P(N): 


[•]  means  "the  largest 
integer  in  •" 


h  =    1    + 

P(N) 

I  •  J  •  K  •  M 

a  =  p(n; 

-    I-J-K 

:-M(h-l) 

i  =  1  + 


A 


J  •  K  •  M 

AA  =  A    J-K-M- (i 
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1  + 


_M 

K-M 


AAA  =  AA  -  K«M(j -1) 


k  =  1  + 


AAA 


M 


m  =  AAA  -  M(k-l) 

In  the  sample  program  contained  in  this  paper,  the  indices 
in  their  original  form  are  stored  in  vector  arrays  and  are 
directly  accessible  throughout  the  computations. 

2.   The  Contraction  with  Respect  to  Vu  . 

^_ him 

This  contraction  takes  into  account  the  possibility 
that  the  stage  m  of  a  mission  may  have  been  noted,  but  that 

the  associated  V    may  have  been  declared  to  be  zero. 

nim   J 


C5j.  C'dlatS   r.nythlii; 


IiVO  i  ViU^ 


'him  "  ' 


the  corresponding  elements  in  the  index  arrays  of  the  mission 
are  eliminated.   This  is  done  prior  to  the  execution  of  the 
game  and  may  therefore  be  referred  to  as  the  basic  contrac- 
tion . 

3.   Working  in  Sub  spaces 

It  can  be  expected  that  most  of  the  x,  .  and  y.-. 
will  be  on  their  boundaries  at  any  stage,  including  the 
approximate  optimal  solution.   This  suggests  the  idea  of 
eliminating  computations  involving  variables  which  have 
fixed  values  either  temporarily  or  throughout  the  process. 

To  do  this,  one  reduces  the  dimensions  of  the 
spaces,  thus  saving  machine  time.   One  can  redefine  the 
space  so  as  to  "freeze"  the  variables  on  the  boundaries 
leaving  the  remainder  free  to  move. 
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Where  in  the  program  should  redefinition  of  the 
Y-space  take  place?   The  minimizer  can  hold  the  subspace 
fixed  and  continue  to  move  in  that  subspace  until  an  ap- 
parent minimum  has  been  reached.   At  this  point  the  al- 
location corresponding  to  this  minimum  has  to  be  checked  for 
optimality  in  the  full  space.   This  will  require  at  least 
one  iteration  through  the  minimization  routine  in  the  full 
space.   After  a  minimum  valid  in  the  full  space  has  been 
obtained,  the  Y-space  is  reduced  to  a  new  subspace.   An 
alternative  approach  is  to  redefine  the  space  after  each 
change  in  the  Y-allocation ,  thus  maintaining  a  continously 
changing  subspace.   The  authors  feel  that  the  latter  will 
enable  the  minimizing  variable  to  stay  in  the  subspace 
icnger  finding  directions  oi  decicdse ,  before  the  ;iccd 
arises  to  employ  the  full  space.   This  idea. was  implemented 
in  the  program. 

The  contractions  for  x  (statement  1871)  and  y  (125) 
eliminate  computations  involving  elements  that  are  on  the 
lower  boundaries.   It  must  be  noted  that  the  contraction 
itself  and  the  complications  caused  by  its  use  take  machine 
time.   Worthwhile  time  savings  in  computation  will  not  be 
realized  until  this  technique  is  applied  to  relatively  large 
scale  problems. 

INDX(N),  N  =  1,...,NNX,  contains  the  indices  of 

the  xhi  >  0,  INDY(N) ,  N  =  1,...,NNY,  contains  the  indices  of 

the  y.-,  >  0.   NNX  and  NNY  are  the  dimensions  of  the  subspaces 
3  K 

for  X  and  Y.   These  index  arrays  are  used  to  control  which 
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variables  are  to  be  involved  in  the  computations.   The  way 
this  is  done  in  practice  can  be  seen  in  the  program  list- 
ing. 

One  remark  concerning  the  DIRECTION  FINDING  ALGO- 
RITHMS, (400)  and  (1400),  may  be  in  order:   these  algorithms 
have  to  be  applied  row  by  row  to  the  matrices   | x.  . |   and 

|y.,  |  j.  In  subspaces,  the  number  of  elements  belonging  to 
3  K 

a  particular  row  varies.   The  outer  do-loop  runs  over  the 

number  of  rows.   To  find  the  numbers  of  elements  per  row 

(these  numbers  serve  as  the  termination  values  of  the  inner 

do-loops)  a  test  on  the  indices  stored  in  INDX  and  INDY  is 

conducted  (402  f f ) ,  (1402  f f) .   The  test  value,  NTEST,  is 

the  index  of  the  last  element  in  the  rows  of   |x,  . I   and 

ii    hi  '  ' 

•  y  •  -  i  i  -    i  esDect ivelv .      Wr> &r<    the    t-~s'    rv<ass&s      313    c  1  ctjigti t s 

I  .  j  K .  ,  .      I  >  r    -    > 

in  the  row  at  hand  have  been  collected  and  the  algorithm 

begins.   Before  the  next  row  is  picked,  NSTART  is  incre- 
mented by  the  number  of  elements  found  in  the  previous  row 
so  that  the  search  through  INDX  or  INDY  always  starts  in  the 
correct  position. 

4 .   Machine  Accuracy  Problems 

The  program  calls  for  frequent  testing  of  floating 
point  numbers.   Throughout  the  development  of  the  program 
these  tests  have  been  sources  of  trouble.   Testing  for 
equality  must  be  strictly  avoided  even  after  -  as  has  been 
done  here  in  various  places  -  variables  close  to  a  fixed 
quantity  have  been  reset  to  that  quantity  (e.g.,  (1050  ff) 
or  (1060  f f ) ) .   Although  the  program  specified  double-pre- 
cision, it  took  extensive  experimentation  to  maintain 
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feasibility,  i.e.,  satisfy  the  side  conditions 

f  xhi  ■  £  yjk  -  l 

to  at  least  the  13tn  decimal  place. 

Of  critical  importance  is  the  accuracy  in  the 
computation  of  the  direction  matrices  y  (GAX)  and  g  (GAY) . 
In  the  neighborhood  of  a  maximum  or  minimum,  the  partials 

F  and  F  are  close  to  zero,  as  are  the  Lagrange  multi- 

x      y  '  &    & 

pliers  (XMU  and  YMU  )  ,  and  the  differences  F  -XMU,F  -YMU 
(SD) .   In  order  to  determine  the  value  of  the  derivative 
game,  the  sum  of  the  squares  of  these  differences  has  to  be 
formed,  an  operation  that  may  very  likely  lead  to  erroneous 
results  which,  in  turn,  carry  over  into  the  computation  of 
y  and  g. 

The  countermeasure  taken  is  to  premultiply  the  SD 
when  they  are  small  by  a  large  number,  (505  f f) ,  (1510  f f)  . 
5 .   Testing  the  Program 

As  the  calling  of  subroutines  is  exceedingly  time 
consuming,  the  present  program  does  not  use  them.   This  made 
debugging  tedious.   The  "standard"  routine,  i.e.,  the  pro- 
gram employing  the  derivative  games  in  the  original  form, 
was  tested  running  a  small  scale  example  where   |x|   was 
(2  X  2)  and   |y|   was  (2  X  4),  making  it  possible  to  hand- 
check  the  computations. 

The  "auxiliary  game"  routine  using  the  B-R  process 
was  debugged  using  the  following  objective  function: 
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.  ^  ~  ~  „  ,1 

-*-  >>  s w  a 


F(x,y)  =  2x1  exp[  -2y1-y2]  +  2x2  exp[  -y1"2y2] 

+  x3  exP  [-y1-y2-y3] 

Subject  to:  L,     x.  =  Zj  y,  =  1 

i        k 

xi>  ^k  *  ° 

with  initial  allocations   x  =  (0,0,1),  y=(0,l,0).   For  a 
discussion  of  this  example  in  light  of  the  algorithm   see 
[4].   The  reason  that  this  can  only  be  solved  through  the 
B-R  routine  lies  in  the  fact  that,  against  the  initial  x, 
any  y  represents  an  exact  minimum,  however,  the  value  of 
the  derivative  game  for  x  is  positive,  yielded  by  y°  = 
(0,  /2/2,  -/2/2). 

Pi  ri  3 1  1  v        pti     <=>y-jiiit->1o     i.i  i  f  n     ^  ^  Q     £"  COUnt"  T2     WHS 

up  where   |x|   was  (4x4),   |y|   was  (5x10).   While  this  does 
not  yet  represent  a  problem  in  high-dimensional  space,  the 
authors  are  confident  that  this  example  provided  a  suffi- 
ciently severe  test  to  demonstrate  the  validity  of  the  pro- 
gram as  written. 

6 .  General  Comments 

The  following  remarks  are  intended  to  facilitate 
the  use  and  modification  of  the  program. 

a.   Initial  Allocation 

The  initial  allocations  are  generated  as  corner 
points  in  the  stationary  part  (99).   It  can  be  expected  that, 
in  the  optimal  solution,  most  of  the  variables  will  be  on 
the  boundaries.   An  initial  point  on  the  boundaries  insures 
that  the  number  of  "absurd"  allocations  is  minimal. 
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If  one  were  to  use  an  interior  point  as  a 
starting  solution,  it  possibly  would  involve  large  numbers 
of  absurd  allocations  (e.g.,  a  submarine  in  an  aircraft 
barrier  patrol) .   The  machine  would  spend  an  enormous 
amount  of  time  reducing  such  allocations  to  zero. 

b.  Distance  Policy 

The  initial  and  re-set  values  for  the  dis- 
tances are  determined  from  the  dimensions  of  the  spaces; 
the  user  may  employ  his  own  rules.   A  compromise  between 
re-set  values  too  large  causing  "overshooting"  and  values 
to  small  causing  "creeping"  should  be  considered.   In  gen- 
eral, "creeping"  is  much  costlier  in  terms  of  machine  time. 
The  policy  for  halving  distances  and  its  rationale  is  out- 
lined in  [ 4  ]  . 

c.  Upper  and  Lower  Bounds 

For  simplicity,  0  and  1  have  been  used  in  the 
program.   Specifying  individual  bounds  a,  •  ,  3i  •  and  a.-,  , 
b.,  does  not  introduce  additional  problems  but  increases  the 
storage  space  required  considerably  (e.g.,  for  y,  two  addi- 
tional arrays  of  the  same  dimension  as  y) .  , 

d.  Modifying  the  Objective  Function 

The  algorithm  as  stated  is  valid  for  concave- 
convex  functions  under  quite  general  conditions.   Though  the 
present  program  has  been  designed  to  handle  a  particular  lin 
ear-exponential  function,  it  is  quite  flexible.   The  objec- 
tive function  mentioned  in  subsection  5  may  serve  to  illus- 
trate this  point: 
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F(x,y)    =    2x1   exp[-2y1-y2]    +    2x^   exp  [-y1"2y2] 

+  x3  exp  [-yry2-y3] 

is  produced  by  a  very  simple  adjustment  of  the  mission 
description : 

h=l    i=l    k=l  j=l  E=2.0 

k=2  j=l  E=1.0 

k=3  j=l  E=0.0   END  OF  MISSION  1 
m=l 

h=l   i=2   k=l   j=l   E=1.0 

k=2    j=l    E=2.0 

k=3    j=l    E=0.0   END  OF  MISSION  2 

m=l 

h=l    i=3   k=l  j=l  E=1.0 

k=2  j=2  E=1.0 

k=3  j=3  E=1.0   END  OF  MISSION  3 
m=l 


/•*    r\ 


Vln  =  2.0 

v131  -  1.0. 

The    associated   C,   .  n  ,-.  ^    are  :  Cnin=Ci1o   =    C11_=1.0 

hj£(k)  111     112     113 

e.  Assigning  Values  V,  . 

&        &  him 

The  values  (of  completing  stage  m  of  mission  i 
for  submarine  of  type  h)  are  relative  measures.   When  as- 
signing V,  .  ,  the  user  should  consider  that  a  submarine  may 
fo    ^   him'  ' 

have  spent  part  of  its  weapons  (resources)  during  stage  m-1 
This  may  reduce  its  operational  capabilities  for  stage  m, 
and  the  value  for  stage  m  should  reflect  this  fact. 

f.  Choosing  p  (RHO) 

The  parameter  p,  mentioned  in  Chapter  II, 
section  B  2.b,  has  to  be  chosen  by  the  user.   It  specifies 


33 


the  accuracy  to  which  Max  <}>(x)  is  to  approximate  the  value 

of  the  game  F(x,y) ,  V.   p  affects  the  y-minimization  by 

controlling  the  accuracy,  e(d),  to  which  the  derivative  game 

D  Max  F(x,y)  =  -VF  -g  has  to  approximate  its  mathematical 
g  x  Y 

value.   The  y-minimization  is  the  most  time  consuming  por- 
tion of  the  process. 

where  6  is  the  diameter  of  the  X-space.   Considering  the 
"worst"  case,  when  d  =  d   =  p/36-L,  (L  is  the  maximum  os- 
cillation of  F  ) ,  it  becomes  apparent  that 

2 

e(do^    =    362-6-L 
is    of   order   of  magnitude    p2*10 

Recall  that  the  conditions  established  for 
the  valididty  of  the  algorithm  are  intended  to  guarantee 
that,  at  the  approximate  optimal  solution,  | § (x) -V| <p . 
Test  runs  of  the  program  seem  to  indicate  that  the  accuracy 
actually  obtained  is  much  higher. 

Although  generally  valid  conclusions  cannot  be 
derived  from  this  observation  the  user  must  be  aware  of  this 
feature  because  he  will  pay  heavily  in  terms  of  machine  time 
when  p  is  unreasonably  small. 
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IV.   SUMMARY 

The  devlopment  of  the  mathematical  theory  underlying 
the  derivative  game  and  the  concave -convex  game  algorithm 
has  only  been  sketched  out  in  this  paper.   Here,  that  al- 
gorithm has  been  employed  in  the  formation  of  a  potentially 
useful  example.   With  programming  techniques  designed  to 
provide  economical  running  in  high  dimensions,  large-scale 
problems  should  be  amenable  to  the  application  of  the  con- 
cave-convex game  algorithm. 
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V.   SUGGESTIONS  FOR  FURTHER  STUDY 

A.    MATHEMATICS 

In  Section  III.C.6.e.,  the  question  of  the  proper  choice 
of  p  was  addressed.   Recall  that 

Max  <f>(x)  "  V|<  p. 
x 

Even  when  the  user  has  based  his  choice  of  a  value  for  p 

on  extensive  experimentation  with  a  program,  he  still  will 

not  know  exactly  how  close  Max  <{>(x)  is  to  the  value  of  the 

x 

game.   The  desirable  state  of  affairs  would  be 

Max  <Kx)  _  V|  =  p 
x 

This  would  require  the  formulation  of  necessary  conditions 

on  the  functions  a   3  and  e . 
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B.    PROGRAMMING 

The  fact  that,  for  the  class  of  games  at  issue,  MaxMinF 

x   y 
=  MinMaxF  can  be  exploited  to  arrive  at  the  neighborhood  of 

y  x 
the  optimal  solution  faster  than  the  present  program  will: 

Start  the  problem  as  MinMaxF.   The  course  of  action  then  is 
reversed  with  considerable  advantages.   The  maximize r  sees 
the  present  y-allocation ;  the  maximization  is  trivial,  as- 
signing as  much  weight  as  possible  to  the  x,  .  with  the 
largest  coefficients  (the  F,(x,y0)),  which  results  in  a 
corner  solution  for  x.   Then  the  B-R  technique  is  employed 
directly  to  F(x,y)  which  will  bring  x  off  the  boundaries 
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again,  and,  after  a  pre-set  number  of  iterations,  will  pro- 
duce an  allocation  not  far  from  the  solution.   At  this  stage 
the  problem  is  reinterpreted  as  MaxMinF  and  solved  in  the 
manner  of  the  present  program. 

Another  feasible  refinement  particularly  useful  in  the 
application  of  the  algorithm   to  objective  functions  linear 

in  both  x  and  y,  i.e.,   F  =  X  x.a..y.,  is  the  idea  of 

13      J    J 
"doubling",  outlined  in  [4].   The  problem  is  treated  as 

MaxMin  and  MinMax  at  the  same  time.   Here  the  value  of  the 

game  is  approached  from  below  and  above  simultaneously  which 

provides  a  stopping  rule  when  the  difference  MaxF(x,y)  -cj)  (x) 

x 

arrives  at  a  pre-specif ied  value. 
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APPENDIX 


FLOWCHART  OF  THE  PROGRAMMED  EXAMPLE 
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COMPUTER  PROGRAM  LISTING 
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